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PRINCIPAL COMPONENT ANALYSIS OF RATINGS 



OF SOME DEVIANT ENGLISH SENTENCES 

Joseph H. Danks and Charles Lewis 
Princeton University 

Abstract 

The comprehension of deviant sentences, a not infrequent demand in 
natural situations, is dependent on several linguistic variables. 
Grammaticalness (g), meaningfulness (M), and familiarity (f) are three 
variables which are potentially such. In order to study the effect of 
violating these variables upon Ss ' responses to deviant sentences, 85 
deviant and 15 correct sentences were assigned to eight groups represent- 
ing all combinations of two values ("correct" or "deviant") on these three 
variables. The 100 sentences were given to four equal groups of Ss (total 
N = 112), who rated each sentence from 0 to 10 on the basis of either 
grammaticalness (G*), meaningfulness (M*), familiarity (F*), or ordinari- 
ness (0*) . The data of the first three groups were then combined into an 
8A by 100 matrix. A principal components analysis was performed on the 
cross-product matrix with a varimax rotation. Four interpretable factors 
emerged, accounting for 89$ of the variability. Factor I was a general 
comprehensibility factor in the factor loadings, related to changes in all 
three variables. However, the familiarity Ss scored highest on Factor I. 
Factors II and III represented G-G* and M-M*, respectively, in both factor 
loadings and factor scores. Factor IV corresponded to the F variable in 
the factor loadings, but was uninterpretable for the factor scores. 
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PRINCIPAL COMPONENT ANALYSIS OF RATINGS 



OF SOME DEVIANT ENGLISH SENTENCES 
Joseph H. Danks and Charles Lewis ^ 

Princeton University 

Although one encounters deviant utterances in several different "natural 
language contexts , " few studies have been devoted to the study of such utter- 
ances . Children, just learning to speak, often produce utterances which are 
deviant according to any reasonable description of adult language. It is, 
of course, true that these utterances follow their own regularities (Braine, 
1963; Brown & Fraser, 1963; Menyuk, 1964). However, if these children's 
utterances are to be comprehended by adults--and, in fact, the large majority 
are comprehensible, at least in part — then the utterances must be processed 

by the adult grammar as deviant. Thus, some subsection of the adult grammar 

2 

must be programmed for the comprehension of such utterances. A second source 



TTow at Kent State University and Dartmouth College, respectively. 

2 

In this paper, we sometimes use "grammar" in a very general sense, 
following Chomsky ( 1965 ). In these cases, "grammar" refers to a complete 
description of the language including phonological, syntactical, and seman- 
tic components. What in common usage is referred to as "grammar" is more or 
less equivalent to "syntax" (cf . instructions to Ss for an example of this 
contrast). If there is a possibility of confusion of the two senses of 
"grammar," the correct sense will be appropriately noted. Thus, "deviant" 
utterances refer to any utterance which violates any of the rules of the 
three components of the grammar. Ultimately, even this formal definition 
of "deviant" reduces to the judgments of native speakers (see Danks, 1968 , 
for a more extended discussion of this point). In this study we will be 
concerned primarily with syntactic and semantic deviances . 

"Comprehension" is also being used in a generic sense. It is the result 
of a complete and successful processing of the utterance through the grammar; 
thus, it includes phonological perception, syntactic interpretation, and 
semantic understanding, "perception," "interpretation," and "understanding" 
do not strictly refer to their common meanings, but are defined as technical 
terms referring to the assigning of a description from the respective com- 
ponents of the grammar. 
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criteria for an ideal grammar be revised so that the grammar generates not 
only all the fully grammatical sentences but also all the deviant ones and 
labels them as such. 

In order to comprehend a deviant utterance, the most parsimonious manner 
in which to process the utterance is to use the "regular” grammar insofar as 
possible, making adjustments only when necessary to account for the deviations 
from regularity. Thus, how one processes deviant utterances will provide 
evidence for possible mechanisms which are operating in the processing of 
normal utterances . The mechanism by which the adjustments are made must 
have a regular link to the ,T normal n grammar (Lakoff, 1965). If, as some 
current psychological theories of language performance suggest (Miller, 1962; 
Miller & Chomsky, 1963), the language user has internalized an analogous 
form of the formal grammar and uses it in the production and comprehension 
of utterances, then there is an additional reason to study the language 
user’s response to deviant and nondeviant utterances in the same situation. 

The few empirical studies to date which have studied any form of deviant 
utterances represent a wide range in experimental sophistication and the 
types of deviances studied (see Danks, 1968 , for a more complete review of 
the linguistic and psychological literature relating to deviant sentences). 

In a polemical discussion. Hill ( 1961 ) sought to rebut Chomsky’s criterion 
that groups of English words can be consistently and agreeably identified 
as gra mm atical or ungrammatical and conducted an informal study using only 
10 sentences and 10 3s. Some of the sentences were formally grammatical, 
some were grammatical but meaningless, and some were ungrammatical. Of the 
10 Ss, 8 were either college professors (English or linguistics) or graduate 
students, 1 was an undergraduate business major, and 1 was a secretary. The 
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Ss were first asked to read aloud all 10 sentences. They were then asked ^ 

to indicate which strings were ungrammatical and which were grammatical. 

j 

Although the conclusions were rather tentative owing to the small select 
sample of both Ss and test items, the results indicated that there was no 
general conception of grammaticalness. Hill also concluded that many non- p 

syntactic variables, e*g., intonation, meaning, punctuation, etc., entered 
into the judgments . 

Maclay and Sleator (i960) presented six sets of sentences varying 
differentially in three dimensions which they called graimnaticalness, mean- 
ingfulness, and ordinariness. The Ss were 57 undergraduates divided into 
three groups. Group I responded to the question "Do these words form a 
grammatical English sentence?" with respect to each string. Group II was 
asked the same question except that "meaning:ful" was substituted for 
"grammatical." And for Group III, the substitution was "ordinary" with the 
same procedure. A final ranking of groups based on the proportion of Ss 
giving a "yes" response matched the predicted rankings based on linguistic 
considerations of the way the strings were constructed. In contrast to Hill 
(1961), the judgments of grammaticalness were relatively independent of mean- 
ingfulness and ordinariness. While no data were presented relevant to 
between-S agreement, there was some indication that the Ss T judgments were 
not in complete agreement. Although the Maclay and Sleator experiment was 
better controlled than that of Hill, there were still methodological problems 
to be solved, e.g., the more precise specification of the dimensions on which 
the sentences were constructed. 

Coleman (1965) constructed 10 series of b sentences, each series having 
4 levels of grammaticalness. The sentences were constructed by a method 
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similar to the theoretical analysis proposed by Miller and Chomsky (1963). 

For the lowest level of grammaticalness words were drawn at random. At 
each succeeding level, word class restrictions were placed on the possible 
choice of words, although within the restrictions the words were randomly 
selected. At the fourth and highest level of grammaticalness used in this 
study, the restrictions were sufficiently severe that, although the words 
were drawn randomly, the strings were very close to being completely gram- 
matical. Ten college fres'unen ranked the k sentences within a.ll 10 series 
separately in order of grammaticalness. Ail series were ranked in the 
predicted direction. Thus, in contrast to Hill’s S s , these Ss were able 
to differentiate degrees of grammaticalness. 

In an extension of Coleman’s work, Tanks (1967) used a subset of 
Coleman's sentences and a different scaling technique. Within-S consistency 
and between-S agreement were high. There was strong evidence that college 
undergraduates can differentiate degrees of grammaticalness as suggested by 
Miller and Chomsky. However, there was no clear differentiation between the 
two lower levels of grammaticalness (the separation is also not clear in 
Coleman's results). Since the word class restrictions of the second lowest 
level were those such as Noun Phrase , from which any word including a verb 
may be ultimately derived, Noun Phrase represents no more a restriction than 
drawing the words at random. The two lower levels are therefore comparable. 
Thus to clarify this situation, the format ives included as word class restric- 
tions must be able immediately to dominate a terminal symbol of the base 
structure in a permissible derivation. Such a qualification excludes Noun 
Phrase, but not Noun . Though the qualification was implicitly followed in 
their examples, it was not stated explicitly by Miller and Chomsky. The 
results of both Coleman and Tanks support this qualification. 
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Marks (1965> 1967 a, 1967 b) has made an extensive study of Ss ' responses 
to varying syntactical distortions- Although his distortions were not in 
general related to a specific linguistic theory, he was able to make pre- 
dictions (which were confirmed) based on a psychological model of sentence 
processing- This model incorporated both linguistic, e-g-, phrase structure 
rules, and psychological notions, e-g-, serial order processing- Marks and 
Miller (1964) studied the interrelation of syntactic and semantic distortions 
on learning of the deviant sentences- Using sentences of the form 

(1) AdJ 1 + + V + Adj 2 + N 2 , 



they effected syntactic distortions by scrambling the word order in several 
ways (anagram sentences)- They assumed that a large measure of the original 
meaning remained in the scrambled form, since either the Ss could recon- 
struct the utterance or induce the general notion from the associative . 
overlap- Distortion of meaning (anomalous sentences) without affecting the 
syntax was accomplished by "diagonalizing" over a set of five sentences 
each of the same form- Thao is, the first word of the first sentence (Wj j ) , 
the second word of the second sentence (W^) > etc*, were combined until 
five new sentences were formed thusly: 



(2) (a) W X1 + W g2 + W 55 + W 41 ^ + W 55 

(b) « 1S + w 23 + + w 45 + w 51 

(c) • • • 

(d.) • • • 

(e) W 15 + W 21 + W 52 + > 
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where W . . = the i_ -th word in the -th sentence. Each sentence was of 
the same syntactical form as (l), but since the topics were mixed the semantic 
structure was effectively destroyed. In addition to the normal anagram and 
anomalous sentences, a fourth set of sentences (word lists) which distorted 
both syntax and semantics was constructed by scrambling the word order of 
(2). The groups of sentences were presented aurally to Ss in a free learning 
situation. The anagram and anomalous sentences produced equal deficits in 
the mean percentage of words correct compared with normal sentences, and, of 
course, the word lists were lowest. However, there were differences between 
the anagram and anomalous sentences in terms of the kinds of errors committed. 
Thus, syntax and semantics can have relatively different effects on Ss ' 
performance when measured via deviant sentences. 

The principal dimension on which deviant utterances have been produced 
has been that of syntax, although semantics and ordinariness also have been 
used. If one is to study how Ss go about comprehending deviant utterances, 
then one of the primary tasks is to identify the relevant variables of the 
sentences. Thus, as many variables as possible were included which might 
potentially have an effect. In this study the sentences to be used were 
chosen to represent all possible combinations of "correct" or "deviant" 
grammatical (G: g or g), meaningful (M: m or m), and familiar (F: 

f or f) sentences. This classification results in eight possible groups 
of sentences. The notion of familiarity was adapted from that of ordinariness 



g denotes a grammatical sentence; g is used to indicate a sentence 
whichJLs not grammatical. Similar remarks hold for m , m , f , f, o, 
and o . Quite obviously none of these variables represent a strictly 
dichotomous construct either empirically or formally; however they were 
constructed as such to facilitate interpretation in this exploratory study. 
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(0: o or o) used by Maclay and Sleator (i960). They define "an ORDINARY 

sentence as one which we [the authors] believe to occur with relatively high 
frequency in English [p. 276 ]." They do not attempt to construct sentences 
corresponding to three of the eight possible types --gmo , gmo , and gmo . 
In the latter two cases, it is clear that a meaningless, yet ordinary, i.e., 
a common occurrence, utterance would be extremely hard to construct. 
Typically, one does not produce utterances which are meaningless under 
sufficiently common conditions that they be called ordinary. However, it 
is conceivable that there are nonmeaningful cliches such that while the 
utterance is familiar, the "purpose" of uttering the sentence is the un- 
commonness or uniqueness of the utterance. For this reason, we shifted 
from the ordinariness variable to that of familiarity. In this way we 
thought it possible to find sentences which might be included in the two 
groups gmf and gmf . We also thought that the third group not studied 
by Maclay and Sleator (gmo or gmf) could be approximated by using syntac- 
tically correct sentences from very esoteric subjects such as recondite 
philosophy or technical science. 

The problem of ordinariness, familiarity, and other variables related 
to frequency of occurrence is fraught with arguments about the importance of 
these variables as constructs. In principle we agree that it is extremely 
improbable that our Ss have heard or seen any of the f sentences before 
and that the objective difference in frequency between f and f sentences 
is so small as to approach zero (if it could be measured; in practice it 
cannot). However, we have included the F variable since, though Ss may 
be wrong, they do have a "feeling" of familiarity and are able to respond 
on the basis of it (see, e.g., Maclay and Sleator, i960). 
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A question implicit in the above discussion to which we will direct 
some attention is the independence of the variables from each other. 
Linguistic theory of both the older structuralist tradition and of the 
newer generative-transformationalist school have emphasized the indepen- 
dence of syntax and semantics. Such a distinction was felt necessary for 
an adequate formal description of language. While the proclaimed indepen- 
dence may be necessary for a formal description, its efficacy for a model 
of the language user would have to be independently demonstrated. Further, 
the independence of F from both G and M has yet to be satisfactorily 
demonstrated, Maclay and Sleator not to the contrary. Although the present 
study is not designed to be the crucial experiment on the question, some 
evidence will be available. 

Thus, the primary purpose of this study was to identify some of the 
variabj.es used by Ss in responding (e.g., by ratings on various criteria) to 
deviant sentences. A secondary purpose was the tentative analysis of the 
interrelations of these variables. 

The procedure devised to measure the effects of variables and their 
interrelations was a principal components analysis of the ratings of the 
individual sentences. Basically, a principal components analysis allows us 
to describe the ratings obtained as a joint function of the instructions 
given and the sentences used. Ideally, we should be able to find three 
values for each sentence which describe its location on hypothetical G , 

M , and F scales. These are known as the factor loadings of the sentences 
In addition, we should be able to find three coefficients for each S which 
describe the extent to which his judgments are based on G* , M* , and F* 
These are known as factor scores. In the basic factor analysis model, the 
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rating given a sentence by a particular S is assumed to be a weighted sum 
of its factor loadings, the weights being the respective factor scores of Ss 

If the language of factor scores does not seem particularly enlighten- 
ing in this context, it is because principal components analysis was first 
developed to describe mental abilities and personality traits. Its use as 
a scaling technique is more recent and dates from Tucker's (i 960 ) work on 
the vector scaling model. 

To actually obtain these factor loadings and factor scores, we extract 
the first several principal components of the raw data matrix. Thes* are 
defined as the best predictors of the raw data in the sense that linear com- 
binations of them account for the maximum variability in the ratings. At 
this point we should mention the essential difference between principal 
components analysis and the related collection of techniques known as factor 
analysis. The former concentrates on explaining overall variability in the 
data, while the latter tries, primarily, to explain covariability, or inter- 
relations. This is not to say that factor analysis does not explain vari- 
ability or that principal components analysis does not explain covariability 
but rather that the emphasis is different in the two approaches. In the 
present study, it was felt that the emphasis should be on reproducing the 
ratings themselves and not the covariances between ratings. 

Method 

Subjects . The Ss were 112 students enrolled in a summer school at 
Rutgers — The State University, New Brunswick, New Jersey. They were paid 
for their services. All were native speakers of English. 
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Sentence materials . One hundred sentences satisfying the criteria for 
one of the eight possible combinations of the three two-valued variables 
discussed above--grammaticalness, meaningfulness , and familiarity--were 
constructed or assembled from other studies. The sentences are listed by 
types in Table 1. The sentences used by Maclay and Sleator (i960) were 
assigned to the appropriate groups; these were the first five sentences in 
groups gmf (1-5), gmf (l6-20), gmf (33-37), gmf (48-52), and gmf 
( 63 - 67 ). A direct translation of ordinariness to familiarity has been 
assumed. The normal sentences, anomalous and anagram string, and the word 
lists used by Marks and Miller (1964) were assigned to the gmf (6-15), 
gmf (38-4-7), gmf (53-62) and gmf (68-77) groups, respectively. The 
distortions of syntax and semantics introduced by Marks and Miller (1964-) 
were highly atypical, so that, with the exception of normal sentences, their 
sentences were all assumed to be unfamiliar. For the remaining 12 sentences 
for group gmf (21-32) appropriate selections were made from examples of 
children's speech reported by Menyuk (1964-). We assumed that these examples 
would provide interesting information about natural language relative to one 
of the specific types of deviations which had prompted this study. 

The group of sentences characterized as gmf ( 78 - 87 ) were derived from 
one of two general sources. The first five were quotes (sometimes with 
slight modifications to shorten the length) from either Kant, Husserl, or 
Merleau-Ponty. The last five were selected (again with some modifications 
of length) from a recent issue of Science . These 10 sentences were supposed 
to be unfamiliar, in general, to this particular population of Ss ; yet all 
were certainly grammatical and meaningful. The last 13 sentences --7 in gmf 
(88-84-) and 6 in gmf ( 95 - 100 ) — were selected from various literary or 
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Table 1 

List of the 100 Sentences Used, Grouped According 
to Their Supposed Construction 



Sentence No * Sentence 

GRAMyiATIGA.L-MEANINGPiJL- FAMILIAR (gmf) 

1 I wish I could write to each of you individually* 

2 The chairman's most important job is timing* 

3 Numerous other countries will be represented* 

4 They finished it yesterday* 

5 He was ready to go* 

6 Rapid flashes augur violent storms* 

7 Pink bouquets emit fragrant odors* 

8 Fatal accidents deter careful drivers* 

9 Melting snows cause sudden floods* 

10 Noisy parties wake sleeping neighbors. 

11 Furry wildcats fight furious battles. 

12 Respectable jewelers give accurate appraisals* 

13 Lighted cigarettes create smoky fumes* 

14 Gallant gentlemen save distressed damsels* 

15 Soapy detergents dissolve greasy stains* 

miGRAMMTICAL-MMNINGFUL-FAMILIAR (gmf) 

16 Not if I have anything to do with it* 

17 Probably, although he may surprise us* 

18 About the time that the new models were shown* 

19 The kind of person who learns to talk with the natives* 

20 In order to get there before they close* 

21 He'll might get in jail* 

22 He'll have to go the doctors* 

23 I want a mild please* 

24 Put the hat* 

25 I know which do they like* 




-15- 



Table 1 (Contd) 



Sentence No- Sentence 



IINGRAMMTICAL-ME1ANINGFUL-FAMILIAR (gmf) (Contd) 

2 6 Who he is kissing? 

27 It was snow yesterday- 

28 Take off it- 

29 She has lots of necklace- 

50 Give me some soaps- 

51 He growed up fast- 

32 Him is a bad boy. 



GRAMyEATICAL-NOIMEANINGFUL-mTAMILIAR (gmf) 



55 'Appointments can now winters generously- 

34 Extra rivers wished casually to cancel off- 

35 Tired windmills hinge a lot of elephants- 

36 Seventeen intuitions ate highly across the right- 

37 During dishing, tolerant marbles remarked off- 

38 Rapid bouquets deter sudden neighbors- 

39 Pink accidents cause sleeping storms- 

40 Fatal snows wake violent odors- 

41 Melting parties augur fragrant drivers- 

42 Noisy flashes emit careful floods- 

43 Furry jewelers create distressed stains- 

44 Respectable cigarettes save greasy battles* 

45 Lighted gentlemen dissolve furious appraisals- 

46 Gallant detergents fight accurate fumes- 

47 Soapy wildcats give smoky damsels- 



UNGRAMMAT ICAL -MEANINGFUL - UNFAMILIAR ( gmf ) 

48 Yesterday, I the child a dog gave- 

49 The with feet aching man came yesterday home- 

50 Get me from the kitchen a big spoon- 
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Table 1 (Contd) 



Sentence No * Sentence 

UNGRAM^ (gmf) (Contd) 

5 1 You can him not understand- 

52 To me was interesting the movie- 

53 Rapid augur violent flashes storms- 

5^ Bouquets pink odors fragrant emit- 

55 Deter drivers accidents fatal careful- 

56 Sudden melting cause floods snows- 

57 Neighbors sleeping noisy wake parties - 

58 Furry fight furious wildcats battles- 

59 Jewelers respectable appraisals accurate give- 

60 Create fumes cigarettes lighted smoky- 

61 Distressed gallant save damsels gentlemen- 

62 Stains greasy soapy dissolve detergents- 

UN GRAMMAT I CAL- N ONMEANINGFUL - UNFAMILIAR (gmf) 

63 A keeps changed very when- 

6b Tables down cod ashes rock under off two syrup 

65 ' Ought cool send had grand the respiratory- 

66 Not off bandage to now lake asked so is were- 

67 Label break to calmed about and- 

68 Rapid deter sudden bouquets neighbors • 

69 Accidents pink storms sleeping cause- 

70 Wake odors snows fatal violent- 

71 Fragrant melting augur drivers parties- 

72 Floods careful noisy emit flashes- 

73 Furry create distressed jewelers stains- 

7^+ Cigarettes respectable battles greasy save- 

75 Dissolve appraisals gentlemen sighted furious 

76 Accurate gallant fight fumes detergents- 

77 Damsels smoky soapy give wildcats- 
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Table 1 (Contd) 



Sentence No - Sentence 

(gmf) 

78 A house is a representation, the transcendental object is unknown- 

79 Natural being is a realm whose existential status is secondary- 

80 The correlate of this consciousness is immanent temporality* 

81 Sensation can be anonymous only because it is incomplete- 

82 Intellectualism remains anterior to the problem of oriented space- 

83 The surface area of silicic acid has been determined by absorption* 

84 Vernier acuities are inseparable for test targets* 

85 The protein network is composed of an amorphous matrix* 

86 The archosaurian ancestors are probably in the ophiocodont group- 

87 The segmentally ganglionated nerve cord was mapped. 

GRAMMATICAL-N01MEANINGPUL- FAMILIAR (gmf) 



88 


A rose is a rose is a rose* 




89 


The medium is the massage - 




90 


The mGme raths outgrabe- 




91 


’ Twas brillig, and the slithy 


toves did gyre and gimble 


92 


May the hair on your* feet grow long* 


93 


The key is in the sunlight at 


the window in the bars- 


9^ 


All mimsy were the borogoves* 





OTGRAMMATICAL-NOIMEANINGFUL-FAMILIAR (gmf) 

95 Lucy in the sky with diamonds - 

96 Rah, rah, siz, boom, bah- 

97 Hutsut ralston on the rillera- 

98 Within you and without you- 

99 In the world, given, flower maddened, Jehovah accept* 

100 Bah, humbug- 
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popular sources, which would, perhaps, he somewhat familiar to the S s . 

Although these sentences can be said to ’mean" something in their respective 
contexts, this meaning is rather obscure, and this even more obscure when 
taken out of context. Some of the sources were Shakespeare, Dickens, 

McLuhan, Tolkein, Allen, Ginsberg, Lewis Carroll, the Beatles, a football 
cheer, and a 1S'30 ' s slang expression. Although it was not expected that all 
of these expressions would be appropriate to their respective groups, it is 
reasonable to assume that enough of them were correctly constructed and 
assigned to provide a valid measure of that group. It is also obvious that 
these groups are not homogeneous with respect to the precise type of devia- 
tions defined by the label of the group. However, we included the many 
different types of deviation because of uncertainty as to which types might 
be the most efficacious. 

Procedure . The 100 sentences were placed in a random order and given 
to four equal groups of 28 Ss, each of which differed only in the instructions. 
All Ss were asked to rate the sentences from 0 to 10, with decimal scaling 
permitted on the basis of only one dimension. The four groups of Ss were 
each asked to rate the sentences on the basis of one of the following labels: 
grammaticalness (G*), meaningfulness (M*), fam iliarity (F*), and ordina riness 
( 0 *)} The Ss were given no additional help in defining these concepts other 
than the single word label. It was implied that they were to define the 
concept for themselves. Examples of the four sets of instructions and the 
random order of sentences are in Appendix A. The four groups of Ss were 



The instructions to the Ss were obviously related to the variables 
identified in the sentences. However, since there is no a priori reason 
to make the correspondence one of equivalence, the two different sets of 
abbreviations have been used for the groups of sentences and groups of Ss. 

O 
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equalized as much as possible in each session. Seven sessions of varying 
size were needed to administer all of the tests. 

Results 

The 112 Ss by 100 sentences raw data matrix is presented in Appendix B 
for reference of anyone wishing to use the data for additional analyses. 

Scale values . The median and semi-interquartile range for each sentence 
were computed for each group of Ss (Table 2). Because of the extreme skew- 
ness of many of the distributions, nonparametric measures are a more reason- 
able estimate of the central tendencies and variabilities. 

The 0* group was included simply to establish a connection between 
Maclay and Sleator's concept of 0 and that of F used here. The 
Spearman correlation between the median scale values across sentences for 
the two groups of Ss was r = 0.93 (corrected for ties). This corre- 

1 — * ’S 

lation was highly significant (t (98) = 25 . 051 * p < .001). Thus* the 

3 

instructions did not have an overall effect on the ratings for F* and 
0* . However, the distinction was introduced specifically to facilitate 
(at least by the intuition of the experimenter) the construction of the 
gmf and gmf sentences. Since these two groups constituted only 13 1 ° of 
the total sample of sentences, it is possible that differences in the ratings 
of these two groups were overshadowed by the similarity of the two concepts 
for the other groups of sentences. Therefore, a Spearman correlation was 
computed between the F and 0 Ss for these two groups of sentences. 

The correlation was also large and significant (r = 0.91* t (ll) = 7-187 > 

2 < .001). Although a supposed distinction between F and 0 was made in 
the construction of sentences, the ratings from these two groups of Ss were 



O 
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Table 2 

Medians (Mdn) and Semi-Interquartile Ranges (SIR) for 
Each Sentence Across the Four Groups of Ss and 
Arranged According to Supposed Construction 



Sentence 

Number 


G* 




M* 


Groups 


°f Ss 

F* 








0* 


Mdn 


SIR 


Mdn 


SIR 


Mdn 


SIR 


Mdn 




SIR 


gmf 


1 


10 . 


1 


10 


0 


10 


2 


10 




1 


2 


10 


1 


10 


1-5 


10 


2 


10 




0 


3 


10 


1 


10 


1 


10 


1 


10 




0.5 


4 


10 


0 


10 


1 


10 


0 


10 




0 


5 


10 


0 


10 


0 


10 


0 


10 




0 


6 


10 


7 


7 


8 


5 


6-5 


5 




6-5 


7 


10 


0 


10 


1 


10 


2 


10 




1-5 


8 


9-25 


2.5 


7.25 


5-5 


8 


6-5 


8. 


5 


3-5 


9 


10 


0*5 


10 


0*5 


10 


l 


10 




0 


10 


10 


0 


10 


0 


10 


l 


10 




0.4 


li 


9-5 


2 


8 


5 


7 


6 


9 




2-5 


12 


10 


0 


10 


0 


10 


1*5 


10 




0 


13 


10 


1 


9 


2.5 


7 


3 


8 




4-5 


14 


.10 


0 


10 


l 


9 


4 


10 




2 


15 


10 


0 


10 


0.5 


10 


2 


10 




0 


& s £ 


16 


7 


2-5 


9 


4 


10 


0 


10 




1 


17 


7 


5 


8.85 


3 


8 


2-5 


9 




3*5 


l8 


3 


3-5 


7 


4 


8.5 


3-5 


8 




5 


19 


5 


3*5 


8.5 


3*5 


8-5 


2*5 


8 




4 


20 


4 


5 


9 


3-25 


10 


1 


8 




4.4 


21 


3-5 


4-5 


7 


5-5 


5 


4 


5 




6 


22 


8 


3-5 


9 


2 


8 


4-5 


9 




4 


23 


4-5 


5 


4 


5 


5 


4-5 


4. 


5 


4.5 


24 


3 


5 


5 


5 


4 


4 


3 




4 


25 


5 


3-5 


6 


5.25 


5 


4 


5- 


5 


5 


26 


6- 75 


4-5 


9 


2*5 


7-5 


4-75 


9 




3-5 


27 


6-5 


5-5 


8 


3-5 


6 


6 


6 




4.5 


28 


4 


7-5 


6*5 


7 


5-5 


6-75 


4 




6 


29 


6 


4-5 


6.8 


5.25 


5*5 


5 


7 




3*5 


30 


6-75 


5-5 


7-5 


3-5 


6 


6 


8 




5 


31 


4 


6 


7-45 


3 


6 


5 


7 




3*5 


32 


4.5 


6 


8.5 


3-5 


6 


3 


6 




4.5 
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33 

34 

35 

36 

37 

38 

39 

14-0 

4i 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

6 1 

62 

63 

64 

65 

66 

67 
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Table 2 (Contd) 



Groups of Ss 

G* M* F* 0* 



Mdn 


SIR 


Mdn 


SIR 


Mdn 


SIR 


Mdn 


SIR 


0 


2 


0 


1-5 


0 


3-5 


1 


3 


3 


6-5 


0-5 


2.5 


0 


2-5 


2 


4-5 


6 


6 


1 


3 


. 1-5 


4 


1-5 


3-5 


3 


5-5 


0 


3-5 


1 


3-5 


2 


3 


1 


3 


0 


1 


0 


4 


1-5 


3-5 


5-5 


7 


2 


4 


2 


5 


3 


3 


7-5 


6-5 


1 


6 


2 


5-5 


3 


4 


7 


6 


2 


5-5 


1-5 


4 


3 


3-5 


3 


6-5 


0 


1-75 


0 


2-5 


1 


3 


6-5 


5*5 


2-5 


5 


1 


3 


2.5 


5.25 


6*5 


8 


2 


4 


0 


4.5 


3 


5-5 


7 


7-5 


0-5 


5 


0 


4 


2 


4.5 


4 


6-5 


1 


4 


0 


3 


2 


4 


7 


7 


1* 5 


3 


2 


4-5 


4 


5-5 


4-5 


7-5 


0 


1 


0 


3-5 


2 


4 



5 


3-5 


6-5 


4.5 


5 


6 


3 


4 


3 


5-5 


7 


5 


4 


6 


3-5 


5 


6.25 


3 


8 


3 


5« 5 


3 


6-5 


4.4 


5 


4 


8 


4 


5 


5 


4.5 


5 


5 


3-75 


8 


3 


6 


4 


6 


5 


2 


3 


3 


5-5 


l 


4 


2 


3- 5 


2 


3-5 


5*5 


5-5 


3 


6 


2 


4-5 


2 


3 


1 


6 


0.5 


4* 5 


2 


5 


2 


5 


6 


3-75 


4 


6-5 


4* 5 


5-5 


2 


4 


6 


5-5 


3 


5 


3 


5 


2 


4 


5 


5 


0 


3 


2 


4 


2 


4 


7 


3 


2-5 


6-5 


3 


4 


0 


2 


2.5 


4-5 


0 


3 


2 


4 


1 


3 


4.5 


5° 5 


I* 5 


5-5 


2 


3 


2 


5 


3*5 


5° 5 


2.5 


5-5 


3 


4.5 



0 


1 


0 


0-5 


0 


1 


0 


1*5 


0 


1-5 


0 


0 


0 


1-5 


0 


2 


0 


1 


0 


1 


0 


2 


1 


3 


0 


0.5 


0 


1 


0 


2 


0 


1 


0 


2 


0 


0 


0 


1 


0 


2 


0 


1 


0 


1*5 


0 


3 


0 


2 
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Table 2 (Contd) 



Groups of Ss 



G# M* F* 0* 

Sentence “ ~ 



Number 


Md In 


SIR 


McLn 


SIR 


McLn 


SIR 


Mdn 


SIR 


gmf (Contd) 

69 


0 


2 


0 


2-75 


0 


2 


0 


1 


70 


0 


2 


0 


2 


0 


2 


1 


2 


71 


1 


3 


0 


1*4 


0 


2-5 


1 


2-5 


72 


0 


2-5 


0 


1-5 


0.5 


4 


0-5 


3 


73 


1 


3 


0 


1 


0 


2 


0-5 


3 


74 


1 


3-5 


0 


1 


0 


0-5 


0-5 


3 


75 


0-5 


2 


0 


1-5 


0 


2 


1 


2-5 


76 


0 


0-5 


0 


2-5 


0 


2-5 


0-5 


2-5 


77 


0 


3 


0 


1 


0 


3 


0-25 


2.5 






78 


8-5 


3-5 


8 


3 


5-5 


6*5 


8.5 


4 


79 


9 


3 


6 


4 


8.5 


4*5 


8 


3*5 


80 


6-5 


4.5 


4 


5-5 


6-5 


5*5 


5*5 


6 


81 


9*5 


2 


8 


3-5 


6 


3*5 


8*5 


5 


82 


10 


1-5 


7 


6-75 


4.5 


5 


6*5 


3*5 


83 


10 


0 


8.5 


5 


7 


5 


9 


2 


84 


9 


3-5 


4.5 


8 


5 


6*5 


8 


7*5 


85 


10 


1 


7-5 


5.25 


7 


3*5 


8 


4 


86 


10 


3 


5 


6-5 


2 


5*5 


7 


6 


87 


10 


2.5 


8 


5 


5-5 


6 


8.5 


4 



88 


8 


4 


8.5 


3*5 


10 


1 


9 


89 


9*5 


3*5 


7*5 


5*6 


6 


5 


5 


90 


5 


9*5 


2*5 


5 


0 


4 


3*5 


91 


7*25 


5*5 


4 


5.85 


8 


8.5 


6 


92 


10 


1 


8 


2*5 


7 


4*5 


8 


93 


5-5 


5*5 


7 


3*5 


5 


6*5 


4*5 


94 


5*5 


8 


1-5 


5 


2 


8.5 


5 



95 

96 

91 

98 

99 
100 



5 4*5 

1 8 

0 2.5 

5-5 5 

1 1.5 

8 5-5 



9-5 4 

7-5 5 

0 1.5 

8 4 

2 5 

8 3 



10 5 

10 1.5 

0 0.5 

9-5 3-5 

1-5 4 

10 1 



7 

9 

0 

5 

2 

9 
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not different in any interesting or significant way. Thus, the instructions 
were not sufficiently detailed for the Ss to make the distinction between 
these two highly similar, but theoretically different concepts. Group 0* 
data have been excluded from the remainder of the analyses for this reason. 

The problem remains as to whether F (or 0) has any reality in the ratings 
of the sentences. 

The medians of the medians for each group of sentences by the four 
groups of Ss are presented in Table 3. Also shown are the number of sentences 
in each group and the number of sentences for which the median rating for 
three groups of Ss (G* > M* , and F*) was in the predicted direction (above 
or below 5)- The two extreme sets of sentences, gmf and gmf , which were 
predicted to be rated highest and lowest on each scale, respectively, were 
in fact so rated. Although there were minor deviations from 10 and 0 on 
various scales for the individual sentences, Ss were able to rate these 
with almost complete agreement (Table 2). 

Three other sets of sentences — gmf , gmf , and gmf--were also rated 
as predicted. In these three sets, the Ss in the various groups were able 
to establish, to some extent, the instructed dimensions and rated the 
sentences appropriately. However, they were not able to divorce themselves 
completely from the other two variables. It is of course possible that the 
theoretical dimensions which were implicitly hypothesized as independent are 
correlated in fact. A possible answer to this question will be found in the 
next section. 

The remaining three sets of sentences were the combinations not used by 
Maclay and Sleator. In all three sets there was a contrast between the M 
F variables. However, in all three instances there was a tie between 
M and F variable — one affecting the other. 



and 

the 
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Table 3 

Median Scale Values for Groups of Sentence Types 
as a Function of the Instructions to Ss 



Sentence 

Type 


Sentences 
at Criteria 


Sentences 
in Group 


G* 


Instructions 

M* 


to Ss 
F* 


0 * 


gmf 


15 


15 


10 


DC 


10 


10 


gmf 


8 


17 


5 


7-5 


6 


7 


gmf 


8 


15 


5-5 


1 


0 


2 


gmf 


9 


15 


2 


6 


3 


3 


gmf 


15 


15 


0 


0 


0 


0-5 


gmf 


2 


10 


9-75 


7-25 


5-75 


8 


gmf 


1 


7 


7.25 


7 


6 


5 


gmf 


0 


6 


2.25 


7-75 


9-75 


6 
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Thus, the sentences were reasonably effective in producing the kinds of 
variation desired. However, there were some discrepancies which may have 
been the result of the use of impure scaling dimensions by the Ss . A more 
precise determination of the effects of each variable was determined by 
the principal components analysis. 

Principal components analysis . It is a basic theorem of principal 
components analysis that the principal components of a data matrix are 
simply described in terms of the characteristic roots and vectors of the 
cross-product matrices. (These cross-product matrices are themselves 
obtained by multiplying the data matrix by its transpose.) The importance 
of this theorem is that several numerical techniques exist for obtaining 
characteristic roots and vectors » Thus, it is possible to compute the 
desired factor scores and factor loadings. 

The data for the three groups of £s (G* , M* , and F*) were combined 
into an 84 Ss by 100 sentences matrix. The characteristic roots and vectors 
were obtained from the Ss by Ss cross-product matrix and the factor scores 
computed. The factor loadings were then determined via the inverse method. 

The characteristic roots are presented in decreasing order of magnitude in 
Table 4 , with the proportion of ’Variability" accounted for by the corres- 
ponding prinicpal component presented in the right-hand column. Preliminary 
study indicated that the first four components were potentially interpretable. 
Thus the components accounted for 89.49 *fo of the variability. To interpret 
the results of this type of analysis, some type of transformation is usually 
applied to the original loadings and scores. Geometrically, this corresponds 
to rotating the axes in the factor space and, as a consequence, the trans- 
formations are called rotations -orthogonal, if the factors remain uncorrelated, 
and oblique, otherwise. 




Table 4 



Characteristic Roots in Decreasing Order of Magnitude 
and Proportion of Variability Accounted For by Each 



Roots 


Proportion of Variability 
Accounted For 


265866*5977 

12887.1825 

8585*3273 

37770498 

2831.9256 

2145.0785 

1873-1806 

1678.1737 

1539-0305 

1274.2579 

1154.6703 

1117-4038 
1060.3375 
1016.1011 
944.7022 
918.5301 
889-3641 
846-3311 
820 . 2197 
750.6917 
13063. 4407 a 


•8173 
•0399 
.0260 
• 0117 
-0088 
-0066 
.0058 
.0052 

• 0048 

.0039 

.0036 

.0035 

.0033 

• 0031 
.0029 
.0028 
.0028 
.0026 
.0025 
.0023 
.0006 



a Sum of remaining 64 roots* 
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Several rotation schemes, both orthogonal and oblique, were attempted 
on the factor matrices. In each case one of the two matrices was rotated 
and the other determined by the transformation matrix. One orthogonal and 
one oblique solution is presented in both of which the sentence factors were 
rotated first and the subject factors determined second. In those rotations 
which first used the subject factors, only three factors were interpretable 
(the obvious result of only three groups of Ss). Those rotations which were 
based on the sentence factors resulted in four interpretable factors (dis- 
cussed in detail below). Thus, only the latter rotations will be discussed. 

'-j 

Orthogonal rotation . A varimax criterion was used for the orthogonal 
rotation since we expected variables to load on more than one factor. The 
median factor loadings and scores for each set of sentences and groups of 
S_s are in Tables 5 a and 5t>, respectively. The complete set of factor load- 
ings and scores is contained in Appendix C. The first sentence factor 
(Factor i) was interpreted as a general comprehensibility factor, relating 
simply to the number of variables — 0, 1, 2, or 5 — which had been violated; 

Mdn gmf = 5.138; Mdn gmf , gmf , and gmf = 1.504; Mdn gmf , gmf , and 
gmf = 0.358; and Mdn gmf = 0.092 (see Figure l). Factor II was a syntactical 
factor indicating whether or not the G variable was deviant: Mdn g = 1.978; 

Mdn g = 0.256.^ The third factor (Factor III) was interpreted as a meaning- 
fulness factor relating to the M variable: Mdn m = 1.564 and Mdn m = 

0.230. Factor IV was an unfamiliarity factor, producing large loadings if 

5 

Mdn g is used to indicate the median of the factor loadings for all 
syntactic sentences without regard to the values of M or F . Similar 
remarks hold for Mdn g , Mdn m , Mdn in , Mdn f , etc. 



ERIC 



-26- 



Table 5 a 

Median Sentence Loadings by Sentence Types 
After Varimax Rotation 



Sentence 

Type 


I 


II 


Factors 

III 


IV 


gmf 


5-158 


1.629 


1.508 


-0.158 


gmf 


1-285 


0-595 


1.740 


0.561 


gmf 


0.047 


2.146 


0.250 


1.285 


gmf 


0.412 


0.115 


2.154 


1.166 


gmf 


0.092 


0.268 


0.119 


1.272 


gmf 


1-515 


2.549 


0.556 


0-224 


gmf 


I.271 


2.076 


0.964 


0-240 


gmf 


5.087 


0.116 


0.461 


O.985 





Table 5 b 

Median Subject Scores by Instruction to Ss After 
Varimax Rotation of Sentences 




Instructions 
to Ss 


Factors 

I II 


III 


TV 


Grammatical 


1-459 1-975 


0-845 


-0.115 


Meaningful 


1.825 0.4l8 


1.942 


0.286 


Familiar 


2.257 0.471 


0.986 


0.115 
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the sentence set was not familiar: Mdn f = 0.2^0 and Mdn f = 1.167. 

Although these interpretations of the factors are supported by the medians 
in Figure 1, the loadings of the individual sentences (Appendix C) and the 
median loadings for the eight groups (Table 5a) were not as well differ- 
entiated on the respective factors. This failure to confirm the predictions 
implicit in the assignment of the sentences to the various groups is due in 
part to the wide variety of sentences included in each group and in part to 
interactions between the variables. The latter point is discussed more 
fully in the Conclusions. 

These interpretations were partially confirmed by the resultant factor 
scores. The G* Ss scored most heavily on Factor II and also had the 
highest score on Factor II, confirming the previous interpretation of 
Factor II as related to G . The M* Ss similarly scored heavily on 
Factor III, also confirming the interpretation of that factor as defining M . 
The F* Ss scored most on Factor I--general comprehensibility — and were the 
highest on that factor. Their score was quite neutral on Factor IV, which 
had previously been interpreted as related to F . This result then explains 
part of the difficulty in interpreting the scale value results . The F* Ss 
do not seem to have the same definition of familiarity as was used to con- 
struct the sentences. Their definition seems to be a composite of the three 
variables, as determined in the comprehensibility factor. 

Oblique rotation . The oblique rotation used was from a program (ROTSIM) 
developed by K. J&reskog. This program rotates to a set of correlated 
factors with the property that the sum of squares of the factor loadings 
which are hypothesized to be small is minimized. 




-29- 



The results of the rotation on the sentence loadings, the resultant 
subject scores, and the factor intercorrelation matrix are presented in 
Tables 6a, 6b, and 6c, respectively. The complete factor loadings and 
factor scores are in Appendix D. The interpretation of these factors was 
the isame as that for the orthogonal rotation (see Figure 2 for medians). 

Factor I — Comprehensibility Mdn gmf = 6.5 12 

Mdn gmf , gmf , g?nf = 5.876 
Mdn gmf , gnf , gmf = 0.669 
Mdn gmf = 1.008 

Factor II — Grammaticalness Mdn g = 4.124 

Mdn g = 0.579 

Factor III — Meaningfulness Mdn m = I .169 

Mdn m = 0.047 



Factor IV — Unfamiliarity 



Mdn f = -0.027 
Mdn f = 2.285 



The factor scores, however, have lost some of their value as supporting 
evidence. Most upsetting is the negligible median loading of the G* Ss 
on Factor II. Of course, the factor scores did not enter into the criteria 
for either rotation, so this lack of support is not completely surprising. 

Finally, it is worth noting that the factor intercorrelation matrix 
gives an extremely confusing picture of the relationships among the factors. 
These interpretations were again confirmed in the subject factors, except 
that the F* Ss score primarily on Factor I (Comprehensibility), as before. 



O 
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Table 6a 

Median Sentence Loadings by Sentence Types After 
Oblique (ROTSIM) Rotation 



Sentence 




Factors 






Type 


i 


II 


hi 


IV 


gmf 


6.512 


5.547 


0.988 


-0.175 


gmf 


2.937 


0.562 


1.651 


CO 

e*- 

OJ 

0 

1 


gmf 


1.190 


4.673 


-0.098 


3.580 


©iif 


0.115 


-0.167 


2.329 


1.002 


gmf 


-1.008 


0.476 


0.059 


2-264 


gmf 


4.819 


5.209 


0.024 


1.512 


gmf 


4.145 


4.422 


0.511 


1.202 


gmf 


2.745 


0.428 


- 0-024 


0-548 






Table 6b 








Median Subject 


Scores by Instruction 


to S_s After 






Oblique (ROTSIM) Rotation of Sentences 




Instructions 


Factors 






to Ss 


I 


II 


III 


IV 


Grammatical 


1-412 


-0.050 


0.450 


0-999 


Meaningful 


2-012 


-1.228 


O.982 


1-464 


Familiar 


2.450 


-1.714 


-o.oo4 


1-649 



Table 6c 

Factor Intercorrelation Matrix Following Oblique 
(ROTSIM) Rotation of Sentences 


Factors 


I 


II 


III 


11 


- 0.847 






hi 


-0.561 


0.455 




IV 


0.800 


-0.810 


-0.265 
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Conclusions 

Wot only were the G and M variables effective in producing pre- 
dictable ratings, but also presumably pure factors of each were obtained. 
These factors provided loadings for comparing individual sentences on a 
given factor. Over the range of syntactic and semantic deviances employed, 
the Ss ' identification of the relevant dimensions, i.e., from the instruc- 
tions, was reasonably close to that assumed in the theoretical construction 
of the sentences . 

The notable exceptions to these occurred in the mf groups. All of 
the major discrepancies from predictions occurred in relation to sentence 
types in which M and F were given opposite values, i.e., mf or mf . 
Also the Ss given instructions to rate on the basis of F* , in contrast to 
G* and M* Ss, did not use the same F factor as was used in the con- 
struction of sentences. The results indicate that F is not independent 

of G and M , especially the latter, but is a second-order variable which 

£ 

is composed of several first-order variables including G and M . 

Let us consider the status of F in a bit more detail. The concept 
of "ordinariness" (0) as used by Maclay and Sleator was adapted to that of 
familiarity. Maclay and Sleator concluded that G , M , and 0 were 
independent. However,- their technique was not adequately designed to provide 
a definite answer to the question of independence. As mentioned earlier, of 
the eight possible types of sentences used here (asstiming F = 0), the three 



The naming of variables as first- and second-order is relative to 
this situation and is not intended to indicate primacy of G and M in 
any absolute sense (although such may in fact be the case). 
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types Maclay and Sleator omitted involved a contrast between M and 0 . 

In four of the five sentence types which they tested, M and 0 had 
equivalent values. Thus, gmo provides their only experimental test of the 
independence of M and 0 . Only 26$ of the Ss said that gmo was meaning- 
ful and 4$ said that it was ordinary. Although these distributions for 

2 

meaningful and ordinary responses are significantly different (using X , 

£ < .01), one can hardly conclude that such results confirm the independence 
of M and 0 because of the small percentage accepting gmo as meaningful. 

In the present experiment, although F was an effective translation of 
0 , the attempt to establish F as uncorrelated with M failed. In sen- 
tences where there was a M-F contrast, there was only a 32$ success rate, 
as measured by the ratings — 12 of 38 sentences failed to match the formula 
used in their construction. When there was no M-F contrast, the success rate 
was 74$ (46 of 62, see Table 3). More detailed study of the ratings in 
Table 2 indicated that while an mf construction was attempted in 13 cases , 
only one case elicited the hypothesized M-F contrast. In the other 12 cases 
M and F were correlated. 

Further evidence for the nonindependence of F is found in the factor 
scores for the Ss whose instructions were to rate on F* . These Ss scored 
most heavily on Factor I: Comprehensibility. With an orthogonal rotation 

their median score was in the middle of the three groups on Factor IV: 
unfamiliarity. And with the oblique rotation their score was highest (the 
opposite was expected) on Factor IV. Thus, the F* Ss were responding to 
some combination of variables which included G and M . 

However, this evidence is not to imply that the F variable had no 
effect whatsoever. There are two points in particular where it did have an 
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effect, Factor XV was interpretable as -unfamiliar ity for the sentence load- 
ings, Although it was not as "clean" as might be desired, it was reasonable 
to interpret it as such. Also, F added its weight to the determination of 
the Factor I loadings , along with G and M • The median factor loadings 
on Factor I for pairs of sentence types, differing only in F , are compared 
in Table 7- In each of the four pairs, the difference is significant. Thus, 
F did have an observable effect, although its effect was mixed with other 
factors • 

The results provided justification for our use of the cross-product 
matrix in the analysis rather than the more common covariance or intercorre- 
lation matrices. The evidence of a general factor corresponding roughly to 
the means of the individual sentence ratings (Factor i) was useful in inter- 
preting the F* S_s and in evaluating the effectiveness of the F variables 
on sentence construction. While this general factor might be regarded as a 
statistical artifact associated with the variable means, in this case it has 
a very real meaning. Since the ratings for all sentences were supposedly on 
the same scale within a group of Ss, the means (or medians) of the sentences 
are directly comparable (as was discussed under Scale values ) , Thus, the 
particular measure of the relation between variables to be used must be 
selected, keeping in mind the properties which were designed into the pro- 
cedure and which are expected in the resultant data. 

As a final point, the orthogonal and oblique rotations should be com- 
pared, In general, there were no great differences among the factor loadings 
for the two solutions. On this basis alone, the simpler factor relationships 
implied by the orthogonal solution should make it preferable to the other. 
Upon noting the loss of the factor scores 1 value and the confusing factor 
relationships associated with the oblique solution, the picture becomes even 
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Table 7 

Comparison of Pairs of Sentence Types, 
Differing Only in f or f 
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0.047 
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0.092 
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clearer. Although certainly not a definitive conclusion on the issue, this 
implied independence of G and M meshes well with the notion in linguistics 
that syntax and semantics are and should be described independently of one 
another . 
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Instructions for Each of the Four Groups— Grammaticalness, Meaningfulness 
Familiarity , and Ordinariness, Respectively-Followed by 
the Random Order of the 100 Sentences 
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Name Age 

Native Language Date 

In this test we will try to measure how well you know the English 
language. On the following pages, there is a list of sentences. Some of 
them will be quite grammatical; others will not be grammatical at all. 

Your task is to rate the sentences on the following scale: 

[ 0 12 3 4 ^ 6 7 8 9 io ] 

un gr ammat i c al gr amma t i c al 

If you think that the sentence is one of the most ungrammatical sentences 
possible, then you should rate it 0. However, if it is completely 
grammatical, then it is to be rated 10- Sentences falling in between 
these two extremes should be given appropriate ratings in the middle range 
of the scale- If you feel that the eleven categories provided are 
insufficient, please feel free to use decimal values between 0 and 10, in 
order to make finer discriminations. Before you make any ratings, read 
through the entire list of sentences, so that you know what kinds of 
sentences to expect* However, each sentence should be rated independently 
of those around it* There is no time limit* Be sure to rate every item* 
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Name Age 

Native Language Late 

In this test we will try to measure how well you know the English 
language- On the following pages, there is a list of sentences* Some of 
them will be quite meaningful; others will not be meaningful at all- Your 
task is to rate the sentences on the following scale: 



[0 12 


3 


k 


5 


6 


7 


8 


9 10 ; 


non-meaningful 














meaningful 



If you think that the sentence is one of the most non-meaningful sentences 
possible, then you should rate it 0- However, if it is completely meaning- 
ful, then it is to be rated 10- Sentences falling in between these two 
extremes should be given appropriate ratings in the middle range of the 
scale- If you feel that the eleven categories provided are insufficient, 
please feel free to use decimal values between 0 and 10, in order to make 
finer discriminations • Before you make any ratings, read through the 
entire list of sentences , so that you know what kinds of sentences to 
expect- However, each sentence should be rated independently of those 
around it- There is no time limit- Be sure to rate every item* 
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Name Age 

Native Language Date 

In this test we will try to measure how well you know the English 
language* On the following pages, there is a list of sentences* Some of 
them will be quite familiar; others will not be familiar at all* Your 
task is to rate the sentences on the following scale: 

[ 01234 5 6 7 8 9 10 ] 

unfamiliar familiar 

If you think that the sentence is one of the most unfamiliar sentences 
possible, then you should rate it 0. However, if it is completely familiar, 
then it is to be rated 10* Sentences falling in between these two extremes 
should be given appropriate ratings in the middle range of the scale* If 
you feel that the eleven categories provided are insufficient, please feel 
free to use decimal values between 0 and 10, in order to make finer dis- 
criminations* Before you make any ratings, read through the entire list 
of sentences, so that you know what kinds of sentences to expect* However, 
each sentence should be rated independently of those around it* There is 
no time limit* Be sure to rate every item* 
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Name Age 

Native Language Date 

In this test we will try to measure how well you know the English 
language- On the following pages , there is a list of sentences- Some of 
them will be quite ordinary; others will not be ordinary at all- Your 
task is to rate the sentences on the following scale: 



[0123 


k 


5 


6 


7 


8 


9 10] 


unordinary 












ordinary 



If you think that the sentence is one of the most unordinary sentences 
possible^ then you should rate it 0- However; if it is completely 
ordinary; then it is to be rated 10- Sentences falling in between 
these two extremes should be given appropriate ratings ir 'lie middle 
range of the scale* If you feel that the eleven categories provided are 
insufficient; please feel free to use decimal values between 0 and 10; in 
order to make finer discriminations- Before you make any ratings; read 
through the entire list of sentences; so that you know what kinds of 
sentences to expect- However; each sentence should be rated independently 
of those around it- There is no time limit- Be sure to rate every item* 
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1 Accidents pink storms sleeping cause* 

2 Bah, humbug* 

3 He'll have to go the doctors* 

4 Lighted cigarettes create smoky fumes* 

5 The archosourian ancestors are probably in the ophiocodont group* 

6 A rose is a rose is a rose* 

7 Natural being is a realm whose existential status is secondary* 

8 Tired windmills hinge a lot of elephants* 

9 The correlate of this consciousness is immanent temporality- 

10 I want a mild please* 

11 Dissolve appraisals gentlemen sighted furious* 

12 Furry wildcats fight furious battles* 

13 Create fumes cigarettes lighted smoky* 

1.4 About the time that the new models were shown* 

15 Bouquets pink odors fragrant emit* 

16 He growed up fast* 

17 Accurate gallant fight fumes detergents* 

18 Distressed gallant save damsels gentlemen* 

19 The protein network is composed of an amorphous matrix* 

20 I know which do they like* 

21 Not off bandage to now lake asked so is were- 

22 The chairman's most important job is timing* 

23 I wish I could write to each of you individually* 

24 Rapid deter sudden bouquets neighbors* 

25 Seventeen intuitions ate highly across the right- 

26 Neighbors sleeping noisy wake parties* 

27. All mimsy were the borogoves- 

28 Gallant detergents fight accurate fumes* 

29 Respectable jewelers give accurate appraisals* 

30 Who he is kissing? 

31 Rapid augur violent flashes storms- 

32 The surface area of silicic acid has been determined by absorption* 

33 In the world, given, flower maddened, Jehovah accept* 

34 Gallant gentlemen save distressed damsels* 

35 Melting snows cause sudden floods* 

36 Damsels smoky soapy give wildcats* 

37 Within you and without you- 

38 The medium is the massage* 

39 Fatal snows wake violent odors* 

40 Stains greasy soapy dissolve detergents* 

41 Probably, although he may surprise us* 

42 Soapy detergents dissolve greasy stains* 

43 The mome raths outgrabe* 

44 Wake odors snows fatal violent* 

45 Yesterday, I the child a dog gave* 

46 Hutsut ralston on the rillera* 

47 Take off it* 

48 Jewelers respectable appraisals accurate give* 

4-9 Ought cool send had grand the respiratory - 

50 Rapid bouquets deter sudden neighbors* 




Continue to next page* 
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52 
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54 ' 
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94 
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99 

100 



The kind of person who learns to talk with the natives* 

Pink bouquets emit fragrant odors* 

He was ready to go* 

May the hair on your feet grow long* 

Pink accidents cause sleeping storms* 

Get me from the kitchen a big spoon* 

You can him not understand* 

Respectable cigarettes save greasy battles* 

The segmentally ganglionated nerve cord was mapped* 

Lucy in the sky with diamonds* 

It was snow yesterday* 

He’ll might get in jail* 

Put the hat* 

During dishing, tolerant marbles remarked off* 

Melting parties augur fragrant drivers* 

Sensation can be anonymous only because it is incomplete* 
Intellectualism remains anterior to the problem of oriented space* 
Furry fight furious wildcats battles* 

Fatal accidents deter careful drivers* 

Fragrant melting augur drivers parties* 

Numerous other countries will be represented* 

They finished it yesterday* 

The with feet aching man came yesterday home* 

Noisy flashes emit careful floods* 

Extra rivers wished casually to cancel off* 

Rapid flashes augur violent storms* 

Label break to calmed about and* 

Vernier acuities are inseparable for test targets* 

Deter drivers accidents fatal careful* 

Floods careful noisy emit flashes* 

A keeps changed very when* 

Give me some soaps* 

A house is a representation, the transcendental object is unknown* 
Tables down cod ashes rock under off two syrup* 

Soapy wild cats give smoky damsels* 

Cigarettes respectable battles greasy save* 

Him is a bad boy* 

Noisy parties wake sleeping neighbors* 

To me was interesting the movie* 

’Tw as brillig, and the slithy toves did gyre and gimble* 

Not if I have anything to do with it* 

Furry jewelers create distressed stains* 

Rah, rah, siz, boom, bah* 

Appointments can now winters generously* 

Furry create distressed jewelers stains* 

Sudden melting cause floods snows* 

Lighted gentlemen dissolve furious appraisals* 

The key is in the sunlight at the window in the bars* 

She has lots of necklace* 

In order to get there before they close* 



Check each of the above sentences to make sure each has been rated, 
but do not change any answers* 



APPENDIX B 

Complete Raw Data Matrix 



G* = 101-128; M* = 201-228; F* = 301-328; 0* = 401-428 
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Orthogonal Rotation — Factor Loadings 
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